Machine Learning for Portfolio Selection Using Structure at the Instance Level

Many combinatorial optimization problems do not have a clear structure, may present many side constraints, and may include subproblems. In addition, different instances within the same domain can have different structure and characteristics. As a consequence it is commonplace that a single algorithm is not the best performer on every problem instance. We consider an algorithm portfolio approach to try to help us select the best algorithm for a given problem instance. Our purpose is twofold: firstly, to show that structure at the instance level is tightly connected to algorithm performance, and secondly to demonstrate that different machine learning and modelling methodologies, specifically Decision Trees (DT), Case Based Reasoning (CBR) and Multinomial Logistic Regression (MLR), can be used to perform effective algorithm portfolio selection. We test our claims by applying the above mentioned techniques to a large set of instances of the Bid Evaluation Problem (BEP) in Combinatorial Auctions. A BEP consists of a Winner Determination Problem (a well-known NP-hard problem best solved by a IP-based approach), and additional temporal information and precedence constraints (which favour a CP-based approach). We solved the BEP instances using a set of different algorithms. We observed that two algorithms; one IP-based and the other a hybrid combining both CP and IP elements, outperformed all the others on all instances. Hence we divided the instances into 2 classes based on which of these 2 algorithms solves them best. In order to perform our analysis we extract a set of structure-based features, that are cheap to determine, from each instance . We apply the Machine Learning methodologies using the extracted features as input data and the best algorithms as prediction classes. Table 1 shows the prediction rates we obtain. They are compared to Weighted Random (WR), a prediction technique based on the frequency distribution of the solution strategy where the strategy that is best most often is suggested most often and DT CBR MLR WR Prediction rate 90% 85% 79% 49%